{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qFdPvlXBOdUN"
      },
      "source": [
        "# Keras Tuner 简介"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MfBg1C5NB3X0"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>     <a target=\"_blank\" href=\"https://tensorflow.google.cn/tutorials/keras/keras_tuner\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\"> 在 TensorFlow.org 上查看</a>   </td>\n",
        "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/tutorials/keras/keras_tuner.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/tutorials/keras/keras_tuner.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 上查看源代码</a></td>\n",
        "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/tutorials/keras/keras_tuner.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a></td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xHxb-dlhMIzW"
      },
      "source": [
        "## 概述\n",
        "\n",
        "Keras Tuner 是一个库，可帮助您为 TensorFlow 程序选择最佳的超参数集。为您的机器学习 (ML) 应用选择正确的超参数集，这一过程称为*超参数调节*或*超调*。\n",
        "\n",
        "超参数是控制训练过程和 ML 模型拓扑的变量。这些变量在训练过程中保持不变，并会直接影响 ML 程序的性能。超参数有两种类型：\n",
        "\n",
        "1. **模型超参数**：影响模型的选择，例如隐藏层的数量和宽度\n",
        "2. **算法超参数**：影响学习算法的速度和质量，例如随机梯度下降 (SGD) 的学习率以及 k 近邻 (KNN) 分类器的近邻数\n",
        "\n",
        "在本教程中，您将使用 Keras Tuner 对图像分类应用执行超调。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MUXex9ctTuDB"
      },
      "source": [
        "## 设置"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "IqR2PQG4ZaZ0"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g83Lwsy-Aq2_"
      },
      "source": [
        "安装并导入 Keras Tuner。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "hpMLpbt9jcO6",
        "outputId": "31f0e33e-670b-4d27-dbe0-a6cdb6e0d3a7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/129.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━\u001b[0m \u001b[32m122.9/129.1 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.1/129.1 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install -q -U keras-tuner"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "_leAIdFKAxAD"
      },
      "outputs": [],
      "source": [
        "import keras_tuner as kt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ReV_UXOgCZvx"
      },
      "source": [
        "## 下载并准备数据集\n",
        "\n",
        "在本教程中，您将使用 Keras Tuner 为某个对 [Fashion MNIST 数据集](https://github.com/zalandoresearch/fashion-mnist)内的服装图像进行分类的机器学习模型找到最佳超参数。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HljH_ENLEdHa"
      },
      "source": [
        "加载数据。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "OHlHs9Wj_PUM",
        "outputId": "12c05430-17b7-42d8-9087-5f81a2fd76f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "29515/29515 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26421880/26421880 [==============================] - 1s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "5148/5148 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4422102/4422102 [==============================] - 1s 0us/step\n"
          ]
        }
      ],
      "source": [
        "(img_train, label_train), (img_test, label_test) = keras.datasets.fashion_mnist.load_data()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "bLVhXs3xrUD0"
      },
      "outputs": [],
      "source": [
        "# Normalize pixel values between 0 and 1\n",
        "img_train = img_train.astype('float32') / 255.0\n",
        "img_test = img_test.astype('float32') / 255.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K5YEL2H2Ax3e"
      },
      "source": [
        "## 定义模型\n",
        "\n",
        "构建用于超调的模型时，除了模型架构之外，还要定义超参数搜索空间。您为超调设置的模型称为*超模型*。\n",
        "\n",
        "您可以通过两种方式定义超模型：\n",
        "\n",
        "- 使用模型构建工具函数\n",
        "- 将 Keras Tuner API 的 `HyperModel` 类子类化\n",
        "\n",
        "您还可以将两个预定义的 <code>HyperModel</code> 类 [HyperXception](https://keras.io/api/keras_tuner/hypermodels/hyper_xception/) 和 [HyperResNet](https://keras.io/api/keras_tuner/hypermodels/hyper_resnet/) 用于计算机视觉应用。\n",
        "\n",
        "在本教程中，您将使用模型构建工具函数来定义图像分类模型。模型构建工具函数将返回已编译的模型，并使用您以内嵌方式定义的超参数对模型进行超调。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "ZQKodC-jtsva"
      },
      "outputs": [],
      "source": [
        "def model_builder(hp):\n",
        "  model = keras.Sequential()\n",
        "  model.add(keras.layers.Flatten(input_shape=(28, 28)))\n",
        "\n",
        "  # Tune the number of units in the first Dense layer\n",
        "  # Choose an optimal value between 32-512\n",
        "  hp_units = hp.Int('units', min_value=32, max_value=512, step=32)\n",
        "  model.add(keras.layers.Dense(units=hp_units, activation='relu'))\n",
        "  model.add(keras.layers.Dense(10))\n",
        "\n",
        "  # Tune the learning rate for the optimizer\n",
        "  # Choose an optimal value from 0.01, 0.001, or 0.0001\n",
        "  hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4])\n",
        "\n",
        "  model.compile(optimizer=keras.optimizers.Adam(learning_rate=hp_learning_rate),\n",
        "                loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
        "                metrics=['accuracy'])\n",
        "\n",
        "  return model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0J1VYw4q3x0b"
      },
      "source": [
        "## 实例化调节器并执行超调\n",
        "\n",
        "实例化调节器以执行超调。Keras Tuner 提供了四种调节器：`RandomSearch`、`Hyperband`、`BayesianOptimization` 和 `Sklearn`。在本教程中，您将使用 [Hyperband](https://arxiv.org/pdf/1603.06560.pdf) 调节器。\n",
        "\n",
        "要实例化 Hyperband 调节器，必须指定超模型、要优化的 `objective` 和要训练的最大周期数 (`max_epochs`)。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "oichQFly6Y46"
      },
      "outputs": [],
      "source": [
        "tuner = kt.Hyperband(model_builder,\n",
        "                     objective='val_accuracy',\n",
        "                     max_epochs=10,\n",
        "                     factor=3,\n",
        "                     directory='my_dir',\n",
        "                     project_name='intro_to_kt')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VaIhhdKf9VtI"
      },
      "source": [
        "Hyperband 调节算法使用自适应资源分配和早停法来快速收敛到高性能模型。该过程采用了体育竞技争冠模式的排除法。算法会将大量模型训练多个周期，并仅将性能最高的一半模型送入下一轮训练。Hyperband 通过计算 1 + log<sub><code>factor</code></sub>(`max_epochs`) 并将其向上舍入到最接近的整数来确定要训练的模型的数量。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cwhBdXx0Ekj8"
      },
      "source": [
        "创建回调以在验证损失达到特定值后提前停止训练。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "WT9IkS9NEjLc"
      },
      "outputs": [],
      "source": [
        "stop_early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UKghEo15Tduy"
      },
      "source": [
        "运行超参数搜索。除了上面的回调外，搜索方法的参数也与 `tf.keras.model.fit` 所用参数相同。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "dSBQcTHF9cKt",
        "outputId": "ae9424a8-ccfd-46f7-c2c2-7835373df8f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Trial 30 Complete [00h 00m 52s]\n",
            "val_accuracy: 0.8615000247955322\n",
            "\n",
            "Best val_accuracy So Far: 0.890999972820282\n",
            "Total elapsed time: 00h 13m 22s\n",
            "\n",
            "The hyperparameter search is complete. The optimal number of units in the first densely-connected\n",
            "layer is 224 and the optimal learning rate for the optimizer\n",
            "is 0.001.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "tuner.search(img_train, label_train, epochs=50, validation_split=0.2, callbacks=[stop_early])\n",
        "\n",
        "# Get the optimal hyperparameters\n",
        "best_hps=tuner.get_best_hyperparameters(num_trials=1)[0]\n",
        "\n",
        "print(f\"\"\"\n",
        "The hyperparameter search is complete. The optimal number of units in the first densely-connected\n",
        "layer is {best_hps.get('units')} and the optimal learning rate for the optimizer\n",
        "is {best_hps.get('learning_rate')}.\n",
        "\"\"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lak_ylf88xBv"
      },
      "source": [
        "## 训练模型\n",
        "\n",
        "使用从搜索中获得的超参数找到训练模型的最佳周期数。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "McO82AXOuxXh",
        "outputId": "c7f9f378-9382-49a5-d1f2-e00ce1dfd0c5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n",
            "1500/1500 [==============================] - 6s 3ms/step - loss: 0.5079 - accuracy: 0.8236 - val_loss: 0.4377 - val_accuracy: 0.8385\n",
            "Epoch 2/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.3748 - accuracy: 0.8647 - val_loss: 0.3720 - val_accuracy: 0.8618\n",
            "Epoch 3/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.3361 - accuracy: 0.8767 - val_loss: 0.3302 - val_accuracy: 0.8813\n",
            "Epoch 4/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.3080 - accuracy: 0.8870 - val_loss: 0.3599 - val_accuracy: 0.8662\n",
            "Epoch 5/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2908 - accuracy: 0.8927 - val_loss: 0.3213 - val_accuracy: 0.8832\n",
            "Epoch 6/50\n",
            "1500/1500 [==============================] - 5s 4ms/step - loss: 0.2783 - accuracy: 0.8962 - val_loss: 0.3268 - val_accuracy: 0.8854\n",
            "Epoch 7/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2638 - accuracy: 0.9026 - val_loss: 0.3127 - val_accuracy: 0.8893\n",
            "Epoch 8/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2526 - accuracy: 0.9043 - val_loss: 0.3163 - val_accuracy: 0.8882\n",
            "Epoch 9/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.2407 - accuracy: 0.9098 - val_loss: 0.3265 - val_accuracy: 0.8834\n",
            "Epoch 10/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2331 - accuracy: 0.9127 - val_loss: 0.3152 - val_accuracy: 0.8861\n",
            "Epoch 11/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2234 - accuracy: 0.9152 - val_loss: 0.3241 - val_accuracy: 0.8884\n",
            "Epoch 12/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.2153 - accuracy: 0.9191 - val_loss: 0.3102 - val_accuracy: 0.8897\n",
            "Epoch 13/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2065 - accuracy: 0.9227 - val_loss: 0.3328 - val_accuracy: 0.8894\n",
            "Epoch 14/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2026 - accuracy: 0.9240 - val_loss: 0.3246 - val_accuracy: 0.8876\n",
            "Epoch 15/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1962 - accuracy: 0.9258 - val_loss: 0.3232 - val_accuracy: 0.8887\n",
            "Epoch 16/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1870 - accuracy: 0.9294 - val_loss: 0.3333 - val_accuracy: 0.8882\n",
            "Epoch 17/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1816 - accuracy: 0.9312 - val_loss: 0.3288 - val_accuracy: 0.8921\n",
            "Epoch 18/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1731 - accuracy: 0.9344 - val_loss: 0.3463 - val_accuracy: 0.8859\n",
            "Epoch 19/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1730 - accuracy: 0.9352 - val_loss: 0.3455 - val_accuracy: 0.8919\n",
            "Epoch 20/50\n",
            "1500/1500 [==============================] - 5s 4ms/step - loss: 0.1625 - accuracy: 0.9389 - val_loss: 0.3274 - val_accuracy: 0.8965\n",
            "Epoch 21/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1580 - accuracy: 0.9414 - val_loss: 0.3555 - val_accuracy: 0.8902\n",
            "Epoch 22/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1554 - accuracy: 0.9414 - val_loss: 0.3596 - val_accuracy: 0.8863\n",
            "Epoch 23/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1524 - accuracy: 0.9435 - val_loss: 0.3603 - val_accuracy: 0.8928\n",
            "Epoch 24/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1483 - accuracy: 0.9449 - val_loss: 0.3706 - val_accuracy: 0.8953\n",
            "Epoch 25/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1415 - accuracy: 0.9461 - val_loss: 0.3832 - val_accuracy: 0.8889\n",
            "Epoch 26/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1387 - accuracy: 0.9477 - val_loss: 0.4026 - val_accuracy: 0.8855\n",
            "Epoch 27/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1366 - accuracy: 0.9484 - val_loss: 0.3793 - val_accuracy: 0.8942\n",
            "Epoch 28/50\n",
            "1500/1500 [==============================] - 5s 4ms/step - loss: 0.1342 - accuracy: 0.9497 - val_loss: 0.3694 - val_accuracy: 0.8957\n",
            "Epoch 29/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1303 - accuracy: 0.9511 - val_loss: 0.3919 - val_accuracy: 0.8923\n",
            "Epoch 30/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1256 - accuracy: 0.9529 - val_loss: 0.3886 - val_accuracy: 0.8931\n",
            "Epoch 31/50\n",
            "1500/1500 [==============================] - 6s 4ms/step - loss: 0.1234 - accuracy: 0.9534 - val_loss: 0.4258 - val_accuracy: 0.8878\n",
            "Epoch 32/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1217 - accuracy: 0.9539 - val_loss: 0.3979 - val_accuracy: 0.8888\n",
            "Epoch 33/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1178 - accuracy: 0.9558 - val_loss: 0.4003 - val_accuracy: 0.8955\n",
            "Epoch 34/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1140 - accuracy: 0.9565 - val_loss: 0.4171 - val_accuracy: 0.8928\n",
            "Epoch 35/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1117 - accuracy: 0.9578 - val_loss: 0.4558 - val_accuracy: 0.8893\n",
            "Epoch 36/50\n",
            "1500/1500 [==============================] - 5s 4ms/step - loss: 0.1095 - accuracy: 0.9587 - val_loss: 0.4320 - val_accuracy: 0.8912\n",
            "Epoch 37/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1080 - accuracy: 0.9593 - val_loss: 0.4346 - val_accuracy: 0.8892\n",
            "Epoch 38/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1043 - accuracy: 0.9614 - val_loss: 0.4380 - val_accuracy: 0.8934\n",
            "Epoch 39/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1036 - accuracy: 0.9611 - val_loss: 0.4437 - val_accuracy: 0.8927\n",
            "Epoch 40/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1002 - accuracy: 0.9626 - val_loss: 0.4493 - val_accuracy: 0.8889\n",
            "Epoch 41/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0966 - accuracy: 0.9647 - val_loss: 0.4420 - val_accuracy: 0.8948\n",
            "Epoch 42/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.0976 - accuracy: 0.9636 - val_loss: 0.4530 - val_accuracy: 0.8944\n",
            "Epoch 43/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0944 - accuracy: 0.9653 - val_loss: 0.4825 - val_accuracy: 0.8923\n",
            "Epoch 44/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.0916 - accuracy: 0.9658 - val_loss: 0.4555 - val_accuracy: 0.8892\n",
            "Epoch 45/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0896 - accuracy: 0.9669 - val_loss: 0.4819 - val_accuracy: 0.8916\n",
            "Epoch 46/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0865 - accuracy: 0.9666 - val_loss: 0.4886 - val_accuracy: 0.8924\n",
            "Epoch 47/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.0867 - accuracy: 0.9671 - val_loss: 0.4930 - val_accuracy: 0.8929\n",
            "Epoch 48/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0878 - accuracy: 0.9679 - val_loss: 0.5247 - val_accuracy: 0.8860\n",
            "Epoch 49/50\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0809 - accuracy: 0.9701 - val_loss: 0.4665 - val_accuracy: 0.8925\n",
            "Epoch 50/50\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.0808 - accuracy: 0.9698 - val_loss: 0.5383 - val_accuracy: 0.8882\n",
            "Best epoch: 20\n"
          ]
        }
      ],
      "source": [
        "# Build the model with the optimal hyperparameters and train it on the data for 50 epochs\n",
        "model = tuner.hypermodel.build(best_hps)\n",
        "history = model.fit(img_train, label_train, epochs=50, validation_split=0.2)\n",
        "\n",
        "val_acc_per_epoch = history.history['val_accuracy']\n",
        "best_epoch = val_acc_per_epoch.index(max(val_acc_per_epoch)) + 1\n",
        "print('Best epoch: %d' % (best_epoch,))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uOTSirSTI3Gp"
      },
      "source": [
        "重新实例化超模型并使用上面的最佳周期数对其进行训练。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "NoiPUEHmMhCe",
        "outputId": "9eb3fcaf-7e63-4cef-e31d-56680ed947c0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "1500/1500 [==============================] - 6s 4ms/step - loss: 0.5051 - accuracy: 0.8211 - val_loss: 0.3977 - val_accuracy: 0.8576\n",
            "Epoch 2/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.3810 - accuracy: 0.8624 - val_loss: 0.3699 - val_accuracy: 0.8663\n",
            "Epoch 3/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.3411 - accuracy: 0.8758 - val_loss: 0.3656 - val_accuracy: 0.8701\n",
            "Epoch 4/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.3096 - accuracy: 0.8859 - val_loss: 0.3353 - val_accuracy: 0.8823\n",
            "Epoch 5/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2937 - accuracy: 0.8910 - val_loss: 0.3443 - val_accuracy: 0.8758\n",
            "Epoch 6/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.2772 - accuracy: 0.8984 - val_loss: 0.3392 - val_accuracy: 0.8806\n",
            "Epoch 7/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2647 - accuracy: 0.9009 - val_loss: 0.3201 - val_accuracy: 0.8848\n",
            "Epoch 8/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2529 - accuracy: 0.9055 - val_loss: 0.3160 - val_accuracy: 0.8867\n",
            "Epoch 9/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.2434 - accuracy: 0.9086 - val_loss: 0.3169 - val_accuracy: 0.8878\n",
            "Epoch 10/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2351 - accuracy: 0.9122 - val_loss: 0.3024 - val_accuracy: 0.8935\n",
            "Epoch 11/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2228 - accuracy: 0.9158 - val_loss: 0.3393 - val_accuracy: 0.8855\n",
            "Epoch 12/20\n",
            "1500/1500 [==============================] - 6s 4ms/step - loss: 0.2175 - accuracy: 0.9195 - val_loss: 0.3095 - val_accuracy: 0.8920\n",
            "Epoch 13/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2077 - accuracy: 0.9223 - val_loss: 0.3346 - val_accuracy: 0.8857\n",
            "Epoch 14/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.2017 - accuracy: 0.9244 - val_loss: 0.3337 - val_accuracy: 0.8876\n",
            "Epoch 15/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1961 - accuracy: 0.9268 - val_loss: 0.3554 - val_accuracy: 0.8842\n",
            "Epoch 16/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1881 - accuracy: 0.9289 - val_loss: 0.3688 - val_accuracy: 0.8762\n",
            "Epoch 17/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1836 - accuracy: 0.9305 - val_loss: 0.3385 - val_accuracy: 0.8914\n",
            "Epoch 18/20\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1769 - accuracy: 0.9339 - val_loss: 0.3447 - val_accuracy: 0.8929\n",
            "Epoch 19/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1724 - accuracy: 0.9355 - val_loss: 0.3434 - val_accuracy: 0.8891\n",
            "Epoch 20/20\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.1678 - accuracy: 0.9378 - val_loss: 0.3563 - val_accuracy: 0.8917\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7ac4e7620f40>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "hypermodel = tuner.hypermodel.build(best_hps)\n",
        "\n",
        "# Retrain the model\n",
        "hypermodel.fit(img_train, label_train, epochs=best_epoch, validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MqU5ZVAaag2v"
      },
      "source": [
        "要完成本教程，请在测试数据上评估超模型。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "9E0BTp9Ealjb",
        "outputId": "b72481c5-a59f-424c-eef2-b5acc71ef460",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 [==============================] - 1s 3ms/step - loss: 0.3940 - accuracy: 0.8849\n",
            "[test loss, test accuracy]: [0.39395737648010254, 0.8848999738693237]\n"
          ]
        }
      ],
      "source": [
        "eval_result = hypermodel.evaluate(img_test, label_test)\n",
        "print(\"[test loss, test accuracy]:\", eval_result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EQRpPHZsz-eC"
      },
      "source": [
        "`my_dir/intro_to_kt` 目录中包含了在超参数搜索期间每次试验（模型配置）运行的详细日志和检查点。如果重新运行超参数搜索，Keras Tuner 将使用这些日志中记录的现有状态来继续搜索。要停用此行为，请在实例化调节器时传递一个附加的 `overwrite = True` 参数。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sKwLOzKpFGAj"
      },
      "source": [
        "## 总结\n",
        "\n",
        "在本教程中，您学习了如何使用 Keras Tuner 调节模型的超参数。要详细了解 Keras Tuner，请查看以下其他资源：\n",
        "\n",
        "- [TensorFlow 博客上的 Keras Tuner](https://blog.tensorflow.org/2020/01/hyperparameter-tuning-with-keras-tuner.html)\n",
        "- [Keras Tuner 网站](https://keras-team.github.io/keras-tuner/)\n",
        "\n",
        "另请查看 TensorBoard 中的 [HParams Dashboard](https://tensorflow.google.cn/tensorboard/hyperparameter_tuning_with_hparams)，以交互方式调节模型超参数。"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "Tce3stUlHN0L"
      ],
      "name": "keras_tuner.ipynb",
      "toc_visible": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}